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(54) GAME DEVICE, SIMULATION APPARATUS AND GAME IMADE DISPLAY METHOD 

(57)Abstract: 

PROBLEM TO BE SOLVED: To enjoy a game with more sense of 
actuality. 

SOLUTION: An actual model 1 can be removed and a camera 1a is 
mounted thereon. A game proceeding means 5 proceeds a game 
corresponding to a state information of the actual model 1 received from 
a state information obtaining means 3, a state information of a 
competitor received from competitor information obtaining means 4, and 
input signals from a control means 2. An action command is output to the 
actual model 1 corresponding to a proceeding state of the game, and an 
imaging command to an imaging means 6 is output. The imaging means 
6 produces images according to the imaging command. A composition 
means 7 composes a picture taken from the camera 1a and a picture 
produced by the imaging means 6 to output to a display device 8. The 
display device 8 shows the picture produced by the imaging means 6. 
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JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1 .This document has been translated by computer. So the translation may not reflect the original precisely. 
2.**** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



DETAILED DESCRIPTION 

[Detailed Description of the Invention] 
[0001] 

[Field of the lnvention]A game device which advances various games because this invention displays a picture 
on a display, A simulation device which makes operation of the model in virtual space experience, And it is 
related with the game image method of presentation which displays a picture on the simulation device using 
the game device and head wearing type display which can obtain high depth perception especially, and a head 
wearing type display about the game image method of presentation for displaying the screen [ game ] 
according to an advancing state. 
[0002] 

[Description of the Prior Art] I n recent years, the game device centering on a picture has spread by 
development of an image processing technique. In this game device, if a picture is displayed on a display and 
it is operated by a controller corresponding to this display image, a display image will change corresponding to 
that ON mosquito. Thereby, a user operates the character displayed on a display and performs a versus 
fighting game etc. 

[0003]ln order to raise presence by a versus fighting game etc., it is effective to display a picture on the display 
of a big screen, but the display of a big screen is expensive. Then, the head wearing type display is developed. 

[0004]A head wearing type display comprises a ring and a display part fundamentally, and it equips with it with 
the feeling which has hung glasses exactly by hanging a ring on a head. And the image inputted from 
computer graphics (CG:Computer Graphics) or a video device is copied out on the portion which is equivalent 
to the lens of glasses. The projected image goes into a user's field of view as a virtual image of 2-meter 
beyond. Thereby, the user can feel as a 50-inch Tsuyoshi Osako screen is seen 2 m ahead. 
[0005] 

[Problem(s) to be Solved by the lnvention]However, since most is the picture generated by computer graphics, 
the character in the conventional game device is superficial, and depth perception may not be obtained. 
Therefore, even if it used the head wearing type display, the limit was among the presence obtained. 
[0006]This invention is made in view of such a point, and is a thing. 
The purpose is to provide the game device which can enjoy a ****** game. 

Other purposes of this invention are to provide the simulation device which can experience the simulation 
which has depth perception more. 
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[0007]Another purpose of this invention is to provide the game image method of presentation which can 

display the picture which has depth perception more. 

[0008] 

[Means for Solving the Problem]ln a game device which displays a picture according to an advancing state of a 
game in order to solve an aforementioned problem in this invention, An optical model which a camera is 
carried and operates according to an operating command, A control means which inputs instructions about 
operation to said optical model, and instructions required for game progress, While advancing a game using a 
state information acquisition means which acquires state information of said optical model, and an entry 
content and said state information from said control means and outputting an operating command according to 
an advancing state of a game to said optical model, A game proceeding means which outputs image display 
instructions according to an advancing state of a game, A picture generation means which generates a picture 
based on image display instructions from said game proceeding means, A game device having a synthesizing 
means which compounds an image photoed with said camera and an image which said picture generation 
means generated, and a display which displays a picture which said synthesizing means combined is 
provided. 

[0009]According to such a game device, state information of an optical model is acquired by state information 
acquisition means. And if a user inputs an operating command of an optical model using a control means, 
based on an operating command and state information, a game will advance by a game proceeding means. 
Then, while an operating command to an optical model is issued and the optical model operates according to 
an advancing state of a game, image display instructions according to an advancing state of a game are 
issued. A picture is generated by picture generation means according to the image display instructions. A 
generated picture is combined with a picture photoed with a camera carried in an optical model by a 
synthesizing means, and is displayed on a display. 

[0010]Since a user can see image composing of a three-dimensional picture which this caught with a actual 
camera, and a picture generated according to an advancing state of a game, a game with higher depth 
perception can be enjoyed. 

[001 1]ln a simulation device which makes operation of a model in virtual space experience in order to solve an 
aforementioned problem, An action calculating means which calculates an action when said model is thing 
according to an entry content from a control means which inputs an operating command to a model, and said 
control means, and outputs image display instructions according to a computed action, A picture generation 
means which generates a picture based on image display instructions from said action control means, It has a 
picture display part which displays a picture which said picture generation means generated, and the function 
to adjust transmission quantity of light from a user's look front, A simulation device having a head wearing type 
display which lays an image from the look front on top of a picture which said picture display part displayed, 
and is made to project to a user's field of view is provided. 

[0012]According to such a simulation device, if a user inputs an operating command using a control means, an 
action when a model is a genuine article will be calculated by action calculating means. Then, a picture 
according to a calculation result of an action is generated by image display means. In a head wearing type 
display, a generated picture is piled up with an image from a user's look front, and is projected to a user's field 
of view. 

[001 3]A user can see by this image composing of a real image ahead of [ actual ] a user, and a picture 
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generated by a simulation, and a simulation with higher depth perception can be experienced. 
[0014]ln the game image method of presentation for displaying a screen [ game ] according to an advancing 
state, in order to solve an aforementioned problem, The game image method of presentation characterized by 
what a picture from a camera carried in an optical model which operates according to an advancing state of a 
game, and a picture generated according to an advancing state of a game are combined, and a combined 
picture is displayed for on a screen of a head wearing type display is provided. 

[0015]According to this game image method of presentation, image composing of a picture from a camera and 
a picture generated according to an advancing state of a game projects on a screen of a head wearing type 
display a user who enjoys a game. 
[0016] 

[Embodiment of the lnvention]Hereafter, an embodiment of the invention is described with reference to 
drawings. Drawing 1 is a principle hneblock diagram of this invention. This game device is a game device 
which performs a pitched-against each other type game. 

[0017]The optical model 1 is movable and the camera 1a is carried. This optical model 1 operates according to 
the input from the game proceeding means 5. Since it is pitched against each other, the control means 2 is an 
input device for operating the function set up virtually, while inputting the operating command about a motion of 
the optical model 1. The functions set up virtually are functions (launch of a missile etc.) for attacking a waging- 
war partner's optical model, for example. 

[0018]The state information acquisition means 3 acquires the state information of the optical model 1. State 
information is information about the position of the optical model 1, or direction. The acquired state information 
is passed to the game proceeding means 5. 

[0019]The waging-war partner information acquisition means 4 acquires a waging-war partner's state 
information sent from a waging-war partner's game device. Specifically, it is the position information on a 
waging-war partner's optical model, etc. The acquired information is passed to the game proceeding means 5. 
[0020]The game proceeding means 5 advances a game according to the state information of the optical model 
1 received from the state information acquisition means 3, the state information of the waging-war partner who 
received from the waging-war partner information acquisition means 4, and the input signal from the control 
means 2. And while outputting the operating command to an optical model according to the advancing state of 
a game, the image generation instructions to the picture generation means 6 are outputted. 
[0021]The picture generation means 6 generates a picture based on image generation instructions. The 
pictures to generate are a part of on-the-spot photo picture recorded beforehand and a picture generated with 
computer graphics. 

[0022]The synthesizing means 7 combines the picture photoed with the camera 1a, and the picture which the 
picture generation means 6 generated, and outputs it to the display 8. The display 8 displays on a screen the 
picture which the picture generation means 6 generated. 

[0023]According to such a game device, if a user controls the optical model 1 using the control means 2, the 
position information on the optical model 1, etc. will be acquired by the state information acquisition means 3, 
and will be sent to the game proceeding means 5. Then, according to state information, such as position 
information on a waging-war partner's optical model, the state information sent from the state information 
acquisition means 3, and the input from the control means 2, a game is advanced by the game proceeding 
means 5. And the picture according to the advancing state of the game is generated by the picture generation 
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means 6, and it is sent to the synthesizing means 7 by it. The picture generated by the picture generation 
means 6 is combined with the picture photoed with the camera 1a, and is sent to the display 8. The image 
composing is displayed with the display 8. 

[0024]The picture generated by this by the three-dimensional image actually caught with the camera 1a 
according to advance of a game is combined, and the user can see the image which is full of that presence 
[ like ] which is holding actual waging war. Under the present circumstances, as the display 8, if a head 
wearing type display is used, depth perception will increase more. 

[0025]Next, an embodiment of the invention is described taking the case of the case where it applies to the 
competition type game system of a tank waging-war vehicle. Drawing 2 is the general drawing of the 
competition type game system of this invention. As shown in a figure, the two game devices 100,200 are 
connected by the communication bus. The optical model 110,210 of the tank is connected to each game 
device 100,200. The optical model 1 10,210 is put on the waging-war space 300, and waging war is held in this 
space. The magnetic field of XYZ shaft orientations is formed in the waging-war space 300. The optical model 
1 10,210 of a tank can recognize a self position and direction correctly by detecting the magnetic field. 
Detection functions, such as a position of the optical model using this magnetic field, are equivalent to the state 
information acquisition means 3 of drawing 1 

[0026]Drawing 3 is a figure showing the composition of a game device. The game device 100 is constituted 
centering on the personal computer 120 (PC) which controls the whole. The optical model 110, the mixer (MIX) 
130, the head wearing type display 140, and the controller 150 are connected to PC120, and these are 
controlled. Based on the control signal from the controller 150, a motion of an optical model is controlled or, 
specifically, the versus fighting game of a tank waging-war vehicle is advanced. And pictures based on the 
advancing state of the versus fighting game, for example, the cartridge of a waging-war partner's tank, etc., 
such as computer graphics, are generated, and it outputs to the mixer 130. The contents of CG to generate are 
changed according to the signal which shows a motion of the head wearing type display 140. Although the 
advancing state of a game changes also with a waging-war partner's state information (position information 
etc.), the state information about a waging-war partner's optical model is sent from the other party's game 
device 200 via a communication bus. Similarly, the state information of the self optical model 1 10 is also 
transmitted to a waging-war partner's game device 200 via a communication bus. 

[0027]The optical model 110 operates according to the signal from PC120. As operation, it is controllable to 
direction of a gun turret and the firing angle of a gun as well as advance or sternway. The optical model 1 1 0 is 
provided with the camera 111 of an ocellus or a stereo. The image which the camera 1 1 1 caught is sent to the 
mixer 130. The function to check states, such as an own position, is carried in the optical model 110. The 
position information on the optical model 1 10 is expressed with the three-dimensional coordinate value (x, y, z) 
in the waging-war space 300. Position information is sent to PC120 as state information with direction (phi) of a 
gun turret, the up-and-down angle (delta) of a gun, etc. 

[0028]The mixer 130 combines the picture sent from the camera 1 1 1 carried in the optical model 110, and the 
picture sent from PC120, and sends it to the head wearing type display 140. 

[0029]The head wearing type display 140 has structure with which a user's head can be equipped. And the 
picture sent from the mixer 130 is displayed on the screen which was able to be provided in the face of the 
user. 

[0030]The controller 1 50 is for inputting the operating command of the optical model 1 1 0. For example, a joy 
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stick etc. are used. From the controller 150, it can be ordered movement of front and rear, right and left of the 
optical model 1 1 0, rotation of a gun turret, the firing angle of a gun, etc. If the button provided in the controller 
150 is pushed, discharge instructions of a gun will be issued. 

[0031] Drawing 4 is a figure showing the internal configuration of PC. PC120 has the following functions. The 
communication control part 121 transmits and receives the information between other game devices. The 
information which specifically sent the information received from other game devices to the game advancing 
part 122, and was sent from the game advancing part 122 is sent to other game devices. 
[0032]The game advancing part 122 is a portion which controls a pitched-against each other type game, and 
the various signals sent from the controller 150, the communication control part 121, the optical model 110, 
and the head wearing type display 140 are based, and it advances a game. The advance result of a game is 
outputted as the operating command to the optical model 110, the image display instructions to the image 
generation part 123, and sound generation instructions to the sound generation part 125. 
[0033]The image generation part 123 generates the picture according to the advancing state of the game 
according to the instructions from the game advancing part 122. CG and an on-the-spot photo picture are used 
for generation of a picture. The on-the-spot photo picture is saved beforehand at the on-the-spot photo picture 
attaching part. When the image generation part 123 uses an on-the-spot photo picture, a required picture is 
taken out from the on-the-spot photo picture attaching part 124, and the all or some pictures are used. An on- 
the-spot photo picture and CG are compounded if needed. 

[0034]The sound generation part 125 generates a sound according to the instructions from the game 
advancing part 122, and outputs the signal of a sound to a head wearing type display. For example, an 
explosion is outputted when the artillery shell discharged toward a partner's optical model hits the mark. 
[0035]Drawing 5 is an outline lineblock diagram showing the composition of a head wearing type display. In the 
head wearing type display 140 of this invention, the liquid crystal display panels (LCD:Liquid CrystalDisplay) 
142a and 142b which the back light 141 was formed and were provided in right and left, respectively are 
illuminated from the upper part. LCD142a and 142b display the picture outputted from the mixer 130. The 
image for left eyes is displayed on LCD142a provided in left-hand side (inside of a figure, right-hand side) to 
the user among LCD142a and 142b, and the image for right eyes is displayed on LCD142b provided in right- 
hand side (inside of a figure, left-hand side) to the user. 

[0036]The image displayed on LCD142a and 142b is reflected by the half mirrors 143a and 143b toward a 
user's front, it is further expanded and reflected by the concave surface half mirrors 144a and 144b, and image 
formation of the reflected image is again carried out to the retina of a user's eyes through the half mirrors 143a 
and 143b (a virtual image is displayed) - it is like. 

[0037]ln this example, the angle of visibility is set as 30 degrees. This will look at the 52 type big screen, when 
it converts into the virtual-image distance of 2 m. And the external world which looks simultaneous with the 
image of LCD142a and 142b by the usual eye can also be seen now by adopting the concave surface half 
mirrors 144a and 144b. The liquid crystal shutter 145 is formed in the point of the concave surface half mirrors 
144a and 144b, and the liquid crystal shutter 145 adjusts a bright dark thing transmitted light amount at it. 
[0038]ln the head wearing type display 140, the position sensing devices 146, such as an acceleration sensor 
and a gyro sensor, are formed, and the motion of the three dimension a user's head is detected to it. The 
signal detected with the position sensing device 146 is sent to PC120. 

[0039]The headphone (graphic display abbreviation) which can be rolled round are formed in the head wearing 
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type display 140, and the sound from the sound generation part 125 is inputted into it. When a user uses the 
head wearing type display 140, headphone are inserted in a user's earhole and a sound is heard. 
[0040]Operation of a game device when the above composition performs the pitched-against each other type 
game of a tank waging-war vehicle is explained below. When starting a game, a user equips his head with the 
head wearing type display 140 first. And the starting command of a game is inputted using the controller 150. 
Then, the picture which the camera 1 1 1 caught, and CG by PC120 or the image by on-the-spot photo is 
compounded by the mixer 130, and is displayed on its own head wearing type display 140. A user operates the 
controller 150, looking at the screen displayed on the head wearing type display 140, and attacks a partner's 
optical model 210. 

[0041] Drawing 8 is a figure showing the example of the display screen of a head wearing type display. In this 
screen 400, some their own guns 41 1 and the waging-war partner's optical model 412 can be seen with the 
background 421. The background 421 puts the picture by CG on an on-the-spot photo picture. Some their own 
guns 41 1 and a waging-war partner's optical model 412 are the pictures which the camera 1 1 1 of the optical 
model 110 caught. 

[0042]Here, if the strength of a gun is adjusted and it discharges after a user operates the controller 1 50 and 
positions his own gun, tank cliff ****** of the waging-war partner will be carried out for an artillery shell. 
[0043]The artillery shell 431 at this time is exaggerated and expressed by CG rather than the usual motion. 
Namely, in PC120, a oneself side generates graphics by making a waging-war partner's coordinates, and 
status into an ON mosquito. As a point of a graphics portion, the locus of the artillery shell of **. itself and a 
waging-war partner is displayed slowly (it is about 1 sec among waging-war partners). 
[0044]** . In order to display expression when an artillery shell hits with reality, a sound effect is generated 
simultaneously, for example, the sound effects (the sound effect generation part 125 generates) of [ if it hits a 
partner's tank ] "DOKAN" -- a burst state is both displayed. 

[0045]** . A background (part) is displayed. Thereby, the locus of the artillery shell of a gun can come to be 
seen, and a versus fighting game can be directed pleasantly. In this embodiment, the information from the 
position sensing device 146 of the head wearing type display 140 responds, and the following control is 
performed. 

[0046]** . Control of direction of a gun turret (vicarious execution of the partial function of the controller 150). 
** Movable [ of the camera device 111] can be been made to carry out with three axes, and control of an 
optical model is enabled only by moving the head. For example, it is being able to save the time and effort of 
operation of a gun turret, and enabling it to concentrate on a game by interlocking direction of the gun turret of 
a tank with direction of the head, and moving it. 

[0047]The game advancing part 122 advances a game, performing the following control, in order to bring more 
the depth perception of this competition type game device close to reality. First, the kinds (a light weight, 
weight, etc.) of artillery shell can be chosen now. The game advancing part 122 changes the advancing state 
of a game according to the kind of artillery shell. For example, a lightweight artillery shell has a locus close to a 
straight line, and destructive power is small if it does not hit on a partner's point. It is difficult for the artillery 
shell of weight for a locus to put in a parabola. Therefore, a ballistic computer may be prepared independently. 
The destructive power of the artillery shell of weight is large. 

[0048]When the cartridge of the artillery shell is carried out, it controls for control of a tank to fall little by little by 
the kind of artillery shell, a place, and the number of times. For example, they are the speed fall of a tank, 
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disappearance of control, the articulation score (resolution, luminosity) fall of a picture, etc. The articulation 
score fall of a picture is adjusted by controlling the liquid crystal shutter 145 of the head wearing type display 
140. If an artillery shell hits a partner's tank, mark will be displayed according to the place which hit, its artillery 
shell will increase, and a subsequent battle will become advantageous. 

[0049]The game device of this invention can extend a variation to application of versus fighting games, such as 
tank object helicopter and naval ship opposite ******, and pet ROBOTTOHE in addition to waging war of the 
illustrated tank waging-war car. If it applies to a pet robot, a virtual experience will be possible in the eye line 
which a pet robot looks at. 

[0050]Although PC is provided for every optical model and he is trying to control by the above-mentioned 
explanation, two or more controllers, optical models, etc. can also be connected, summarized and controlled to 
one computer. 

[0051]The head wearing type display used for the versus fighting game device of this invention is provided with 
see-through functions (liquid crystal shutter etc.). The depth perception that its optical model is fighting with the 
waging-war partner in the virtual image can be obtained by piling up and seeing the virtual image displayed on 
a head wearing type display, spacing and looking at the optical model of itself and a partner with this see- 
through function. At this time, the optical model which carries a camera is unnecessary. That is, the image 
which entered from a user's front with the see-through function of the head wearing type display is used 
instead of the picture caught with the camera of the optical model of the above-mentioned game device. Since 
a picture is optically combined in a head wearing type display, the function of the mixer 130 of dr awing 3 is 
also unnecessary. 

[0052]The game device of this invention can consider the still more nearly following applications. First, since 
virtual reality is experienced, the game device of this invention is applicable to a virtual reality experience 
system. This is a system which is formed in the head wearing type display 140 and which realizes experience 
like the attraction of a theme park using the liquid crystal shutter 145 which controls the transmitted light 
amount from the external world. 

[0053]That is, the game advancing part 122 controls the liquid crystal shutter 145 as follows. Drawing 7 is a 
flow chart which shows the liquid crystal shutter control procedure for a virtual reality experience system. 
Hereafter, along with a step number, the control procedure of the liquid crystal shutter by the game advancing 
part 122 is explained. 

[S1] At the time of a game start A liquid crystal shutter is made completely open. 

[S2] It goes into the world of a game by closing a liquid crystal shutter little by little, carrying out fade-in 

gradually. 

[S3] A liquid crystal shutter is closed thoroughly and the picture in the world of a game is displayed by LCD. 
[S4] It escapes from the world of a game, carrying out fade-out of the liquid crystal shutter gradually by 
opening little by little. 

[S5] A liquid crystal shutter is opened thoroughly and the real world is shown to a user. 

[0054]Thus, by controlling a liquid crystal shutter, experience like the attraction of a theme park is realizable. 

Other operations are the same as that of the case where a pitched-against each other type game is performed. 

[0055]ln a versus fighting game device, movable distance of its optical model can also be enlarged by putting 
the partner of a versus fighting game on real environment. That is, although it was not able to be pitched 
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against each other in the conventional game device except the space beforehand set up by a program, in the 
game device concerning this invention, an optical model can be arranged at arbitrary places and it can be 
pitched against each other at the place. Therefore, for example, if an optical model is carried out out in the 
fields, the behavior range of an optical model can be enlarged dramatically. 

[0056]The simulation function of a versus fighting game is also applicable to training. For example, when 
applying the simulation function of this invention to crane operation, the position sensing device of a crane 
operation simulator + head wearing type display + head wearing type display is prepared. A crane operation 
simulator is equivalent to PC and the controller in the above-mentioned game device. The game advancing 
part in PC is transposed to the action calculating means which calculates the action (the action of the load 
hung by the crane is also included) of a crane by a crane operation simulator. It is only saying that a game 
element is not included in calculation contents, and this action calculating means is substantially unchanging 
with the function of the game advancing part of a game device. 

[0057]On a head wearing type display, the scene which is in sight when an operator sits on a driver's seat is 
displayed. "It is CG altogether", "CG+ on-the-spot photo", "the image from the camera with which CG+ model 
was equipped", etc. are possible for the display at this time. And the scene which is in sight on a head wearing 
type display to compensate for operation of the crane operation simulator by an operator is changed. Thereby, 
the simulation of crane operation comes made simply. 

[0058]ln addition, it is applicable to the simulation of a scene which cannot use a light airplane, a glider, and 
other full-scale simulations. These can also be made into a game as they are. 

[0059]The contents of the processing capability which the above-mentioned PC performs can be described to 
the program recorded on the recording medium which can be read by computer. And the above-mentioned 
processing is realized by the computer by executing this program by computer. As a recording medium which 
can be read, there are a magnetic recording medium, semiconductor memory, etc. by computer. In circulating 
a commercial scene, store and circulate a program to portability type recording media, such as CD-ROM 
(Compact Disk Read Only Memory) and a floppy disk, or, It stores in the storage device connected via the 
network, and can also transmit to other computers through a network. When performing by computer, the 
program is stored in the hard disk drive in a computer, etc., and it loads to main memory and performs. 
[0060] 

[Effect of the lnvention]As explained above, in the game device of this invention. Since the picture which the 
camera carried in the optical model which operates according to the advancing state of a game caught, and 
the picture generated according to the advancing state of a game are combined and it was made to display, 
the user of this game device can enjoy a game with depth perception. 

[0061]ln the simulation device of this invention, since the real image ahead of a user and the picture generated 
by the simulation are piled up and shown, a virtual image is laid on top of an actual image, and a simulation 
with a higher touch of reality can be experienced. 

[0062]The user who and looks at the displayed picture combining the picture which the camera carried in the 
optical model caught in the game image method of presentation of this invention, and the picture generated 
according to the advancing state of a game, and having made it display can get high depth perception. [ the 
user ] 
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EFFECT OF THE INVENTION 

[Effect of the lnvention]As explained above, in the game device of this invention. Since the picture which the 
camera carried in the optical model which operates according to the advancing state of a game caught, and 
the picture generated according to the advancing state of a game are combined and it was made to display, 
the user of this game device can enjoy a game with depth perception. 

[0061]ln the simulation device of this invention, since the real image ahead of a user and the picture generated 
by the simulation are piled up and shown, a virtual image is laid on top of an actual image, and a simulation 
with a higher touch of reality can be experienced. 

[0062]The user who and looks at the displayed picture combining the picture which the camera carried in the 
optical model caught in the game image method of presentation of this invention, and the picture generated 
according to the advancing state of a game, and having made it display can get high depth perception. [ the 
user] 
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MEANS 

[Means for Solving the Problem]ln a game device which displays a picture according to an advancing state of a 
game in order to solve an aforementioned problem in this invention, An optical model which a camera is 
carried and operates according to an operating command, A control means which inputs instructions about 
operation to said optical model, and instructions required for game progress, While advancing a game using a 
state information acquisition means which acquires state information of said optical model, and an entry 
content and said state information from said control means and outputting an operating command according to 
an advancing state of a game to said optical model, A game proceeding means which outputs image display 
instructions according to an advancing state of a game, A picture generation means which generates a picture 
based on image display instructions from said game proceeding means, A game device having a synthesizing 
means which compounds an image photoed with said camera and an image which said picture generation 
means generated, and a display which displays a picture which said synthesizing means combined is 
provided. 

[0009]According to such a game device, state information of an optical model is acquired by state information 
acquisition means. And if a user inputs an operating command of an optical model using a control means, 
based on an operating command and state information, a game will advance by a game proceeding means. 
Then, while an operating command to an optical model is issued and the optical model operates according to 
an advancing state of a game, image display instructions according to an advancing state of a game are 
issued. A picture is generated by picture generation means according to the image display instructions. A 
generated picture is combined with a picture photoed with a camera carried in an optical model by a 
synthesizing means, and is displayed on a display. 

[0010]Since a user can see image composing of a three-dimensional picture which this caught with a actual 
camera, and a picture generated according to an advancing state of a game, a game with higher depth 
perception can be enjoyed. 

[001 1 ]l n a simulation device which makes operation of a model in virtual space experience in order to solve an 
aforementioned problem, An action calculating means which calculates an action when said model is thing 
according to an entry content from a control means which inputs an operating command to a model, and said 
control means, and outputs image display instructions according to a computed action, A picture generation 
means which generates a picture based on image display instructions from said action control means, It has a 
picture display part which displays a picture which said picture generation means generated, and the function 
to adjust transmission quantity of light from a user's look front, A simulation device having a head wearing type 
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display which lays an image from the look front on top of a picture which said picture display part displayed, 
and is made to project to a user's field of view is provided. 

[0012]According to such a simulation device, if a user inputs an operating command using a control means, an 
action when a model is a genuine article will be calculated by action calculating means. Then, a picture 
according to a calculation result of an action is generated by image display means. In a head wearing type 
display, a generated picture is piled up with an image from a user's look front, and is projected to a user's field 
of view. 

[001 3]A user can see by this image composing of a real image ahead of [ actual ] a user, and a picture 
generated by a simulation, and a simulation with higher depth perception can be experienced. 
[0014]ln the game image method of presentation for displaying a screen [ game ] according to an advancing 
state, in order to solve an aforementioned problem, The game image method of presentation characterized by 
what a picture from a camera carried in an optical model which operates according to an advancing state of a 
game, and a picture generated according to an advancing state of a game are combined, and a combined 
picture is displayed for on a screen of a head wearing type display is provided. 

[0015]According to this game image method of presentation, image composing of a picture from a camera and 
a picture generated according to an advancing state of a game projects on a screen of a head wearing type 
display a user who enjoys a game. 
[0016] 

[Embodiment of the lnvention]Hereafter, an embodiment of the invention is described with reference to 
drawings. Drawing 1 is a principle lineblock diagram of this invention. This game device is a game device 
which performs a pitched-against each other type game. 

[0017]The optical model 1 is movable and the camera 1a is carried. This optical model 1 operates according to 
the input from the game proceeding means 5. Since it is pitched against each other, the control means 2 is an 
input device for operating the function set up virtually, while inputting the operating command about a motion of 
the optical model 1. The functions set up virtually are functions (launch of a missile etc.) for attacking a waging- 
war partner's optical model, for example. 

[0018]The state information acquisition means 3 acquires the state information of the optical model 1. State 
information is information about the position of the optical model 1, or direction. The acquired state information 
is passed to the game proceeding means 5. 

[0019]The waging-war partner information acquisition means 4 acquires a waging-war partner's state 
information sent from a waging-war partner's game device. Specifically, it is the position information on a 
waging-war partner's optical model, etc. The acquired information is passed to the game proceeding means 5. 
[0020]The game proceeding means 5 advances a game according to the state information of the optical model 
1 received from the state information acquisition means 3, the state information of the waging-war partner who 
received from the waging-war partner information acquisition means 4, and the input signal from the control 
means 2. And while outputting the operating command to an optical model according to the advancing state of 
a game, the image generation instructions to the picture generation means 6 are outputted. 
[0021]The picture generation means 6 generates a picture based on image generation instructions. The 
pictures to generate are a part of on-the-spot photo picture recorded beforehand and a picture generated with 
computer graphics. 

[0022]The synthesizing means 7 combines the picture photoed with the camera 1a, and the picture which the 

http://ww4ipcU.inpit.go.ip/cgi-bin/tran_web_cgi_eiie?atw_u=htto 11/18/2009 



JP,11-309269,A [MEANS] Page 3 of 7 

picture generation means 6 generated, and outputs it to the display 8. The display 8 displays on a screen the 
picture which the picture generation means 6 generated. 

[0023]According to such a game device, if a user controls the optical model 1 using the control means 2, the 
position information on the optical model 1, etc. will be acquired by the state information acquisition means 3, 
and will be sent to the game proceeding means 5. Then, according to state information, such as position 
information on a waging-war partner's optical model, the state information sent from the state information 
acquisition means 3, and the input from the control means 2, a game is advanced by the game proceeding 
means 5. And the picture according to the advancing state of the game is generated by the picture generation 
means 6, and it is sent to the synthesizing means 7 by it. The picture generated by the picture generation 
means 6 is combined with the picture photoed with the camera 1a, and is sent to the display 8. The image 
composing is displayed with the display 8. 

[0024]The picture generated by this by the three-dimensional image actually caught with the camera 1a 
according to advance of a game is combined, and the user can see the image which is full of that presence 
[ like ] which is holding actual waging war. Under the present circumstances, as the display 8, if a head 
wearing type display is used, depth perception will increase more. 

[0025]Next, an embodiment of the invention is described taking the case of the case where it applies to the 
competition type game system of a tank waging-war vehicle. Drawing 2 is the general drawing of the 
competition type game system of this invention. As shown in a figure, the two game devices 100,200 are 
connected by the communication bus. The optical model 110,210 of the tank is connected to each game 
device 100,200. The optical model 1 10,210 is put on the waging-war space 300, and waging war is held in this 
space. The magnetic field of XYZ shaft orientations is formed in the waging-war space 300. The optical model 
1 10,210 of a tank can recognize a self position and direction correctly by detecting the magnetic field. 
Detection functions, such as a position of the optical model using this magnetic field, are equivalent to the state 
information acquisition means 3 of drawing 1. 

[0026]Drawing 3 is a figure showing the composition of a game device. The game device 100 is constituted 
centering on the personal computer 120 (PC) which controls the whole. The optical model 110, the mixer (MIX) 
130, the head wearing type display 140, and the controller 150 are connected to PC120, and these are 
controlled. Based on the control signal from the controller 150, a motion of an optical model is controlled or, 
specifically, the versus fighting game of a tank waging-war vehicle is advanced. And pictures based on the 
advancing state of the versus fighting game, for example, the cartridge of a waging-war partner's tank, etc., 
such as computer graphics, are generated, and it outputs to the mixer 130. The contents of CG to generate are 
changed according to the signal which shows a motion of the head wearing type display 140. Although the 
advancing state of a game changes also with a waging-war partner's state information (position information 
etc.), the state information about a waging-war partner's optical model is sent from the other party's game 
device 200 via a communication bus. Similarly, the state information of the self optical model 1 1 0 is also 
transmitted to a waging-war partner's game device 200 via a communication bus. 

[0027]The optical model 110 operates according to the signal from PC120. As operation, it is controllable to 
direction of a gun turret and the firing angle of a gun as well as advance or sternway. The optical model 1 1 0 is 
provided with the camera 111 of an ocellus or a stereo. The image which the camera 1 1 1 caught is sent to the 
mixer 130. The function to check states, such as an own position, is carried in the optical model 110. The 
position information on the optical model 1 10 is expressed with the three-dimensional coordinate value (x, y, z) 
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in the waging-war space 300. Position information is sent to PC120 as state information with direction (phi) of a 
gun turret, the up-and-down angle (delta) of a gun, etc. 

[0028]The mixer 130 combines the picture sent from the camera 1 1 1 carried in the optical model 110, and the 
picture sent from PC120, and sends it to the head wearing type display 140. 

[0029]The head wearing type display 140 has structure with which a user's head can be equipped. And the 
picture sent from the mixer 130 is displayed on the screen which was able to be provided in the face of the 
user. 

[0030]The controller 150 is for inputting the operating command of the optical model 110. For example, a joy 
stick etc. are used. From the controller 150, it can be ordered movement of front and rear, right and left of the 
optical model 1 1 0, rotation of a gun turret, the firing angle of a gun, etc. If the button provided in the controller 
150 is pushed, discharge instructions of a gun will be issued. 

[0031]Drawing 4 is a figure showing the internal configuration of PC. PC120 has the following functions. The 
communication control part 121 transmits and receives the information between other game devices. The 
information which specifically sent the information received from other game devices to the game advancing 
part 122, and was sent from the game advancing part 122 is sent to other game devices. 
[0032]The game advancing part 122 is a portion which controls a pitched-against each other type game, and 
the various signals sent from the controller 150, the communication control part 121, the optical model 110, 
and the head wearing type display 140 are based, and it advances a game. The advance result of a game is 
outputted as the operating command to the optical model 1 10, the image display instructions to the image 
generation part 123, and sound generation instructions to the sound generation part 125. 
[0033]The image generation part 123 generates the picture according to the advancing state of the game 
according to the instructions from the game advancing part 122. CG and an on-the-spot photo picture are used 
for generation of a picture. The on-the-spot photo picture is saved beforehand at the on-the-spot photo picture 
attaching part. When the image generation part 123 uses an on-the-spot photo picture, a required picture is 
taken out from the on-the-spot photo picture attaching part 124, and the all or some pictures are used. An on- 
the-spot photo picture and CG are compounded if needed. 

[0034]The sound generation part 125 generates a sound according to the instructions from the game 
advancing part 122, and outputs the signal of a sound to a head wearing type display. For example, an 
explosion is outputted when the artillery shell discharged toward a partner's optical model hits the mark. 
[0035] Drawing 5 is an outline lineblock diagram showing the composition of a head wearing type display. In the 
head wearing type display 140 of this invention, the liquid crystal display panels (LCD:Liquid CrystalDisplay) 
142a and 142b which the back light 141 was formed and were provided in right and left, respectively are 
illuminated from the upper part. LCD142a and 142b display the picture outputted from the mixer 130. The 
image for left eyes is displayed on LCD142a provided in left-hand side (inside of a figure, right-hand side) to 
the user among LCD142a and 142b, and the image for right eyes is displayed on LCD142b provided in right- 
hand side (inside of a figure, left-hand side) to the user. 

[0036]The image displayed on LCD142a and 142b is reflected by the half mirrors 143a and 143b toward a 
user's front, it is further expanded and reflected by the concave surface half mirrors 144a and 144b, and image 
formation of the reflected image is again carried out to the retina of a user's eyes through the half mirrors 143a 
and 143b (a virtual image is displayed) -- it is like. 

[0037]ln this example, the angle of visibility is set as 30 degrees. This will look at the 52 type big screen, when 
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it converts into the virtual-image distance of 2 m. And the external world which looks simultaneous with the 
image of LCD142a and 142b by the usual eye can also be seen now by adopting the concave surface half 
mirrors 144a and 144b. The liquid crystal shutter 145 is formed in the point of the concave surface half mirrors 
144a and 144b, and the liquid crystal shutter 145 adjusts a bright dark thing transmitted light amount at it. 
[0038]ln the head wearing type display 140, the position sensing devices 146, such as an acceleration sensor 
and a gyro sensor, are formed, and the motion of the three dimension a user's head is detected to it. The 
signal detected with the position sensing device 146 is sent to PC120. 

[0039]The headphone (graphic display abbreviation) which can be rolled round are formed in the head wearing 
type display 140, and the sound from the sound generation part 125 is inputted into it. When a user uses the 
head wearing type display 140, headphone are inserted in a user's earhole and a sound is heard. 
[0040]Operation of a game device when the above composition performs the pitched-against each other type 
game of a tank waging-war vehicle is explained below. When starting a game, a user equips his head with the 
head wearing type display 140 first. And the starting command of a game is inputted using the controller 150. 
Then, the picture which the camera 1 1 1 caught, and CG by PC120 or the image by on-the-spot photo is 
compounded by the mixer 130, and is displayed on its own head wearing type display 140. A user operates the 
controller 150, looking at the screen displayed on the head wearing type display 140, and attacks a partner's 
optical model 210. 

[0041] Drawing 6 is a figure showing the example of the display screen of a head wearing type display. In this 
screen 400, some their own guns 41 1 and the waging-war partner's optical model 412 can be seen with the 
background 421. The background 421 puts the picture by CG on an on-the-spot photo picture. Some their own 
guns 41 1 and a waging-war partner's optical model 412 are the pictures which the camera 1 1 1 of the optical 
model 110 caught. 

[0042]Here, if the strength of a gun is adjusted and it discharges after a user operates the controller 1 50 and 
positions his own gun, tank cliff ****** of the waging-war partner will be carried out for an artillery shell. 
[0043]The artillery shell 431 at this time is exaggerated and expressed by CG rather than the usual motion. 
Namely, in PC120, a oneself side generates graphics by making a waging-war partner's coordinates, and 
status into an ON mosquito. As a point of a graphics portion, the locus of the artillery shell of **. itself and a 
waging-war partner is displayed slowly (it is about 1 sec among waging-war partners). 
[0044]** . In order to display expression when an artillery shell hits with reality, a sound effect is generated 
simultaneously, for example, the sound effects (the sound effect generation part 125 generates) of [ if it hits a 
partner's tank ] "DOKAN" - a burst state is both displayed. 

[0045]** . A background (part) is displayed. Thereby, the locus of the artillery shell of a gun can come to be 
seen, and a versus fighting game can be directed pleasantly. In this embodiment, the information from the 
position sensing device 146 of the head wearing type display 140 responds, and the following control is 
performed. 

[0046]** . Control of direction of a gun turret (vicarious execution of the partial function of the controller 150). 
** Movable [ of the camera device 111] can be been made to carry out with three axes, and control of an 
optical model is enabled only by moving the head. For example, it is being able to save the time and effort of 
operation of a gun turret, and enabling it to concentrate on a game by interlocking direction of the gun turret of 
a tank with direction of the head, and moving it. 

[0047]The game advancing part 122 advances a game, performing the following control, in order to bring more 
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the depth perception of this competition type game device close to reality. First, the kinds (a light weight, 
weight, etc.) of artillery shell can be chosen now. The game advancing part 122 changes the advancing state 
of a game according to the kind of artillery shell. For example, a lightweight artillery shell has a locus close to a 
straight line, and destructive power is small if it does not hit on a partner's point. It is difficult for the artillery 
shell of weight for a locus to put in a parabola. Therefore, a ballistic computer may be prepared independently. 
The destructive power of the artillery shell of weight is large. 

[0048]When the cartridge of the artillery shell is carried out, it controls for control of a tank to fall little by little by 
the kind of artillery shell, a place, and the number of times. For example, they are the speed fall of a tank, 
disappearance of control, the articulation score (resolution, luminosity) fall of a picture, etc. The articulation 
score fall of a picture is adjusted by controlling the liquid crystal shutter 145 of the head wearing type display 
140. If an artillery shell hits a partner's tank, mark will be displayed according to the place which hit, its artillery 
shell will increase, and a subsequent battle will become advantageous. 

[0049]The game device of this invention can extend a variation to application of versus fighting games, such as 
tank object helicopter and naval ship opposite ******, and pet ROBOTTOHE in addition to waging war of the 
illustrated tank waging-war car. If it applies to a pet robot, a virtual experience will be possible in the eye line 
which a pet robot looks at. 

[0050]Although PC is provided for every optical model and he is trying to control by the above-mentioned 
explanation, two or more controllers, optical models, etc. can also be connected, summarized and controlled to 
one computer. 

[0051 ]The head wearing type display used for the versus fighting game device of this invention is provided with 
see-through functions (liquid crystal shutter etc.). The depth perception that its optical model is fighting with the 
waging-war partner in the virtual image can be obtained by piling up and seeing the virtual image displayed on 
a head wearing type display, spacing and looking at the optical model of itself and a partner with this see- 
through function. At this time, the optical model which carries a camera is unnecessary. That is, the image 
which entered from a user's front with the see-through function of the head wearing type display is used 
instead of the picture caught with the camera of the optical model of the above-mentioned game device. Since 
a picture is optically combined in a head wearing type display, the function of the mixer 130 of drawing 3 is 
also unnecessary. 

[0052]The game device of this invention can consider the still more nearly following applications. First, since 
virtual reality is experienced, the game device of this invention is applicable to a virtual reality experience 
system. This is a system which is formed in the head wearing type display 140 and which realizes experience 
like the attraction of a theme park using the liquid crystal shutter 145 which controls the transmitted light 
amount from the external world. 

[0053]That is, the game advancing part 122 controls the liquid crystal shutter 145 as follows. Drawing 7 is a 
flow chart which shows the liquid crystal shutter control procedure for a virtual reality experience system. 
Hereafter, along with a step number, the control procedure of the liquid crystal shutter by the game advancing 
part 122 is explained. 

[S1] At the time of a game start A liquid crystal shutter is made completely open. 

[S2] It goes into the world of a game by closing a liquid crystal shutter little by little, carrying out fade-in 

gradually. 

[S3] A liquid crystal shutter is closed thoroughly and the picture in the world of a game is displayed by LCD. 
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[S4] It escapes from the world of a game, carrying out fade-out of the liquid crystal shutter gradually by 
opening little by little. 

[S5] A liquid crystal shutter is opened thoroughly and the real world is shown to a user. 

[0054]Thus, by controlling a liquid crystal shutter, experience like the attraction of a theme park is realizable. 

Other operations are the same as that of the case where a pitched-against each other type game is performed. 

[0055]ln a versus fighting game device, movable distance of its optical model can also be enlarged by putting 
the partner of a versus fighting game on real environment. That is, although it was not able to be pitched 
against each other in the conventional game device except the space beforehand set up by a program, in the 
game device concerning this invention, an optical model can be arranged at arbitrary places and it can be 
pitched against each other at the place. Therefore, for example, if an optical model is carried out out in the 
fields, the behavior range of an optical model can be enlarged dramatically. 

[0056]The simulation function of a versus fighting game is also applicable to training. For example, when 
applying the simulation function of this invention to crane operation, the position sensing device of a crane 
operation simulator + head wearing type display + head wearing type display is prepared. A crane operation 
simulator is equivalent to PC and the controller in the above-mentioned game device. The game advancing 
part in PC is transposed to the action calculating means which calculates the action (the action of the load 
hung by the crane is also included) of a crane by a crane operation simulator. It is only saying that a game 
element is not included in calculation contents, and this action calculating means is substantially unchanging 
with the function of the game advancing part of a game device. 

[0057]On a head wearing type display, the scene which is in sight when an operator sits on a driver's seat is 
displayed. "It is CG altogether", "CG+ on-the-spot photo", "the image from the camera with which CG+ model 
was equipped", etc. are possible for the display at this time. And the scene which is in sight on a head wearing 
type display to compensate for operation of the crane operation simulator by an operator is changed. Thereby, 
the simulation of crane operation comes made simply. 

[0058]ln addition, it is applicable to the simulation of a scene which cannot use a light airplane, a glider, and 
other full-scale simulations. These can also be made into a game as they are. 

[0059]The contents of the processing capability which the above-mentioned PC performs can be described to 
the program recorded on the recording medium which can be read by computer. And the above-mentioned 
processing is realized by the computer by executing this program by computer. As a recording medium which 
can be read, there are a magnetic recording medium, semiconductor memory, etc. by computer. In circulating 
a commercial scene, store and circulate a program to portability type recording media, such as CD-ROM 
(Compact Disk Read Only Memory) and a floppy disk, or, It stores in the storage device connected via the 
network, and can also transmit to other computers through a network. When performing by computer, the 
program is stored in the hard disk drive in a computer, etc., and it loads to main memory and performs. 
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DESCRIPTION OF DRAWINGS 

[Brief Description of the Drawings] 

[ Drawing 1] lt is a principle lineblock diagram of this invention. 

[Drawing 2] It is the general drawing of the competition type game system of this invention. 
[Drawing 3] It is a figure showing the composition of a game device. 
[Drawing 4] lt is a figure showing the internal configuration of PC. 

[Drawing S] It is an outline lineblock diagram showing the composition of a head wearing type display. 
[Drawing 6]lt is a figure showing the example of the display screen of a head wearing type display. 
[Prawjng__7|lt is a flow chart which shows the liquid crystal shutter control procedure for a virtual reality 
experience system. 
[Description of Notations] 

1 [ -- A state information acquisition means, 4 / - A waging-war partner information acquisition means, 5 / -- A 
game proceeding means, 6 / -- A picture generation means, 7 / -- A synthesizing means, 8 / -- Displaying 
means ] -- An optical model, 1a -- A camera, 2 -- A control means, 3 
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